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METHOD AND APPARATUS FOR RECOGNIZING A 
STRUCTURE TO BE APPLIED ONTO A SUBSTRATE 

WITH MULTIPLE CAMERAS 

[0001] The present application relates to and claims priority from European 
Application Serial No. PCT/EP2004/0 14698 filed December 23, 2004, titled "METHOD 
FOR RECOGNISING A STRUCTURE TO BE APPLIED TO A SUBSTRATE, WITH 
THE AID OF SEVERAL CAMERAS AND DEVICE THEREFOR", the complete 
subject matter of which is hereby expressly incorporated in its entirety. 

[0002] The present invention relates to a method for recognizing a structure to be 
applied onto a substrate with at least one or more cameras, and a corresponding 
apparatus. 

[0003] For recognizing a structure to be applied onto a substrate it has been 
customary to carry out optical measurements. Frequently various systems for fully 
automatic testing of the structure, including adhesive and sealing agent lines, are used. 
For this purpose, multiple video cameras are directed at the structure to be recognized. In 
addition, an illumination module serving to generate a contrast-rich camera image is 
required. 

[0004] In order to be able to monitor an adhesive line or adhesive trail while the 
structure is being applied, it is necessary to teach-in a reference adhesive trail (i.e., to 
have the camera or cameras scan the reference adhesive tail) in order to calculate 
corresponding parameters on which the assessment of the applied adhesive trails are 
subsequently based. 

[0005] Conventionally there was a necessity in the teaching-in of a reference 
adhesive trail for each camera to individually scan the adhesive trail in order to obtain 
camera images for all positions. When using three cameras, the reference adhesive trail 
had to be scanned thrice in sequence and the three different sequences of images of the 
three cameras had to be assigned. A disadvantage with this method is that the 
parameterization of the reference adhesive trail being cumbersome and time consuming 
and can lead to high inaccuracy. 

[0006] Moreover, there is a need for a method to recognize a structure to be 
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applied onto a substrate using at least one or more cameras, and to monitor an application 
structure or adhesive trail at a high accuracy and speed while the structure is being 
applied. 

[0007] It is therefore the present invention is to provide a method for recognizing 
a structure to be applied onto a substrate for at least one or more cameras. The method 
facilitates rapid start-up at a high accuracy and permits quick teach-in of the reference 
adhesive trail. 

[0008] Moreover, an object of the present invention is to provide a method for 
recognizing a structure to be applied onto a substrate using at least one or more cameras. 
The method monitors an application structure or adhesive trail at a high accuracy and 
speed while the structure is being applied. 

[0009] Moreover, an object of the present invention is to provide a suitable 
apparatus for carrying out the method according to the invention. 

[0010] In the method and apparatus according to the invention, a reference 
application structure is taught-in. In addition, further applied application structures or 
adhesive trails are compared to the reference application structure for assessment. The 
application facility according to the invention (i.e., the application facility with cameras 
or as a kinematic inversion) can be moved or the substrate can be moved. 

[0011] The method facilitates by means of teaching-in a reference application 
structure, and recording and storing of the images of all cameras in a sequence of images. 
The cameras take a single scan of said reference application structure. The single scan 
reduces the time of the start-up of an apparatus because of a short teach-in time. 

[0012] According to a method of the invention for recognizing a structure to be 
applied onto a substrate, (e.g., preferably an adhesive line or adhesive trail) at least one 
camera, (e.g., in particular multiple cameras), is used during the scanning for assessment 
of the structure and is processed in the form of an optical representation. Each camera 
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records a strip of the image to form a sequence of images, and the image recording rate is 
increased in line with the data reduction achieved by recording only a strip of the image. 
An advantage is that the method is highly accurate and can be fully automated. This 
allows the adhesive trail to be monitored in parallel or simultaneous to the application of 
the adhesive trail. 

[0013] Furthermore, an advantage is for each camera to record a strip of the 
image to form a part of a sequence of images to minimize the data to be included in a 
calculation. Because less data is included in the calculation and because of the high 
image recording rate, recording small partial sections of the whole adhesive trail to be 

recorded (e.g., between 1 mm and 3 mm) is feasible. The adhesive trail being a type of 

» 

vector chain can be followed automatically in the individual images. 

[0014] Moreover, another advantage is to increase the image recording rate in line 
with the data reduction achieved by recording only a strip of the image. In particular by 
using the partial scanning technique, a higher frame rate is attained. Thus, capturing 
images synchronously and in parallel by all cameras at the same time is feasible. The 
image recording therefore proceeds at defined fixed time intervals and is independent of 
robot control. 

[0015] According to the invention, the image strips of the individual cameras are 
joined to form a single image. Thus, the respective images of the individual cameras are 
assigned accordingly independent of location and can be recorded and processed 
synchronously. In addition, errors are prevented in the teach-in because the images of the 
individual cameras are always assigned with a correct time and a location. 

[0016] According to a preferred embodiment, each camera uses only 
approximately a quarter of the image lines as image strips, thus quadrupling the image 
recording rate. Moreover, an advantage of the parameterization of the sequence of 
images obtained from the reference application structure is that the results from a single 
image recording run of all cameras is automatically performed by means of a one-time 
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external indication or external marking (e.g. by means of a mouse click) of the reference 
application structure. The external marking is used for comparison to an applied 
adhesive trail. In particular, the robot travel path, the robot travel time, the direction, the 
width and the quality of the adhesive trail are used for parameterization. This simple 
parameterization procedure reduces the risk of maloperation or incorrect 
parameterization. The system can be also be operated by personnel with low 
qualifications. Thus, the parameterization generates a vectorization of the entire adhesive 
trail. The vectorization provides for reliable and automatic monitoring of the adhesive 
trail because of the high image recording rate. In addition, the vectorization results in a 
switch of the sensor head requiring recalibration or new calibration without having to 
teach-in again the adhesive trail. 

[0017] According to a preferred embodiment of the invention, an assessment 
function, in particular a fuzzy assessment is used to analyze the adhesive agent track or 
adhesive trail. In particular the width of the pair of edges comprising the right and the 
left edge of the adhesive trail, the mean gray scale value of the projected gray scale value 
profile between the pair of edges, the edge contrast, and the progression of position are 
included in the calculation by means of the assessment function. As a result, the adhesive 
trail can be described accurately such that the adhesive trail can be recognized 
automatically in a reliable fashion. 

[0018] Moreover, an advantage for the edge of the adhesive trail to be determined 
on a surrounding track or orbit (e.g., an essentially circular track) or circular line in order 
to capture any progression of the adhesive trail around the application facility in a defined 
area. The adhesive trail progresses within the surrounding track, which can be elliptical, 
polygonal, or approximately circular. 

[0019] According to a preferred embodiment, the center of the circular line or the 
surrounding track essentially coincides with the site from which the adhesive emanates to 
form the adhesive trail. Each camera monitors at least one segment of the circle formed 
by the circular line. 
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[0020] Errors at the transition from one camera to the next can be reduced by 
having each camera monitor at least one overlapping area jointly with at least one 
adjacent camera. 

[0021] It is particularly advantageous for the angle values of the circular line 
ranging from 0 to 360° form a global coordinate system of the individual cameras. A 
segment of the circular line is assigned to the images of the individual cameras. As a 
result, the progression of the adhesive trail can be followed by at least one active camera, 
whereby statements concerning the entire adhesive trail as well as the position or 
progression of the adhesive trail can be made by relatively simple means. 

[0022] According to a preferred embodiment, a first camera covers a range of 
angles ranging from -10° to 130°, a second camera covers a range of angles from 1 10° to 
250°, and a third camera covers a range of angles from 230° to 10°, where three cameras 
are being used. 

[0023] Moreover, during the progression of the adhesive trail, an advantage is to 
automatically switch from one camera to the next when the adhesive trail progresses from 
the segment of a circular line of one camera via the overlapping area to the segment of a 
circular line of a different camera. As a result, it is feasible to reliably follow the 
progression of the track or position of the track, and these are predictable accordingly. 
Therefore, fully automatic switching between neighboring cameras can occur such that 
the parameterization times are reduced. 

[0024] Moreover, an advantage of light emitting diode (LED) illumination means 
is the color that provides a suitable contrast to the color of the application structure is 
used for illumination. A color of the light is selected such that, according to the principle 
of complementary colors, a maximal contrast between the substrate and the adhesive trail 
results. An advantage of the invention is to use infrared LEDs or ultraviolet LEDs (UV 
LEDs) that emit red, green and blue light, in particular upon the use of red-green-blue 
LEDs (RGB LEDs). As a consequence of using LEDs, the sensor design can be switched 
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to a corresponding adhesive color without further reconfiguration. 

[0025] If triple colored LEDs (e.g., red, green, and blue) are used, the most 
suitable color combined can be generated for optimal contrast. 

[0026] According to a further development of the invention, the LEDs are 
flashed, in particular, by utilizing pulses of current (e.g., ranging from 1.0 to 0.01 ms) 
applied to the diodes. Flashing the diodes, obtains focused images of the adhesive trail, 
while the sensor scans over the substrate. 

[0027] Moreover, for three-dimensional positional correction for the application 
facility is performed by means of the stereometry procedure with regard to positional 
tolerances of the individual components or tolerances of the joining seams and provides 
an advantage for the reference contour or a feature to be determined by at least two 
cameras. 

[0028] Another advantage is if the two cameras record the substrate, a section of 
the component, or one or more components in the form of a full image or large image, 
whereby the full images or large images of the two cameras comprise an overlapping area 
in a leading direction. The three-dimensional recognition of the reference contour 
position results in the overlapping area used for gross adjustment of the application 
facility prior to applying the structure. Thus, corresponding correction values are 
transmitted to the application facility or the robot in order to shift the application 
facility's or robot's coordinate system for the application of the adhesive agent. 

[0029] If a projection is made onto the area of the reference contour for a three- 
dimensional analysis, (e.g., if one or more laser lines are applied to the substrate in the 
form of a projection), then the three-dimensional analysis of the profile with regard to the 
height and contour of arbitrary components can be facilitated even though this is not 
analyzable by common image processing without an additional projection. 

[0030] An advantage of the invention is the individual cameras are calibrated in 
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order to assign the angle assignment of the individual cameras according to a circular 
caliper. In particular a circular arc of the calibrating device with marker points at 0°, 
120°, and 240° for three cameras is used. The marker points allow a global coordinate 
system to be used with regard to the angle assignment for the individual cameras on the 
circular caliper around the application facility in order to simplify the processing by 
software. 

[0031] Moreover, the distance of the application facility from a feature of the 
component (e.g., from edges of the component, or holes or breakthroughs) can be 
measured in order to carry out a positional test of the applied structure. A line-shaped 
gray scale value scan or line-shaped calipers are used for distance measurement such that 
any number of reference markers can be placed along the circular arc of the calibrating 
device. Image processing is not exclusively limited to the camera image, in which the 
adhesive trail test is carried out. It is therefore not necessary for the adhesive trail and 
suitable reference features to be visualized in the same camera image, which has a 
particular advantage with regard to the parallel processing of three camera images. 

[0032] The present invention also provides an apparatus for recognizing a 
structure to be applied onto a substrate, preferably an adhesive line or adhesive trail, 
whereby at least one illumination module and one sensor unit are provided. The sensor 
unit includes at least one or more cameras that are arranged around the facility for 
applying the structure. Each camera is directed at the facility for applying the structure. 
By this means the travel path of the facility over a substrate or a travel path of the 
substrate relative to the application facility to be monitored at all times in all directions 
by means of directing the cameras at the application facility is feasible. 

[0033] An advantage is if the axial longitudinal axis of the individual cameras 
essentially intersects (e.g., in the direction of view or along the axial longitudinal axis of 
the application facility), a narrow area around the application facility can be monitored at 
suitable resolution and high image recording rate. 
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[0034] According to a preferred embodiment, individual cameras, in particular 3 
cameras, are arranged at equal distances from each other in the direction of the 
circumference. 

[0035] The individual cameras are circuited such that the images of the cameras 
are stored in a sequence of images such that these images can be recorded synchronously 
and in parallel as well as in an assigned fashion. 

[0036] According to a development of an invention, one or more cameras form a 
circular caliper whose center is formed by the application facility of the structure. Thus, 
one or more circular calipers can be used to facilitate the determination of the edge of the 
adhesive trail on a circular line. 

[0037] According to a preferred embodiment, the individual cameras comprise an 
overlapping area of at least 10° each relative to the adjacent camera. This overlapping 
area facilitates fully automatic switching between neighboring cameras when the 
adhesive trail progresses from the monitoring area of one camera to the next. Because 
the selection of the camera is not bound to the robot position or a time component, but 
rather always refers to the actual inspection results. For instance, the inspection results 
are based on the arrangement, the orbit, the circular line of the circular caliper, or the 
global coordinate system formed thereby. 

[0038] Moreover, an advantage is for the illumination module to comprise light 
emitting diodes (LEDs), in particular infrared LEDs, ultra-violet LEDs (UV LEDs) or 
Red-Green-Blue LEDs (RGB LEDs). 

[0039] Moreover, an advantage is the use of a calibrating disc with individual 

form elements for calibrating the individual cameras for the assignment of the angle 

assignment. The form elements comprise, in particular, an angle distance of essentially 

10°, which allows for assignment of a scaling factor, an angle assignment, and a center 

and a radius of the search circle for the individual cameras. According to the invention, 
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the calibrating disc comprises at least three marker sites that are arranged in a circular arc 
of the calibrating disc (e.g. essentially located at 0°, 120 °, and 240 °), in order to calibrate 
the three cameras. 

[0040] If a projection facility projecting one or more features (e.g., in particular 
strips) onto the substrate for the three-dimensional analysis is provided on the application 
facility, arbitrary components can be used for correction or adjustment of the application 
facility prior to applying the structure. 

[0041] According to a preferred embodiment, the projection facility emits one or 
more laser lines for a three-dimensional profile analysis. Arranging at least two 
projection facilities around the application facility facilitates a gap-free three-dimensional 
analysis around the application facility. Means of image processing, whereby the 
analyses of a sealing agent height and a sealing agent contour, as well as a position and a 
width, can be performed according to the principle of triangulation. 

[0042] Advantages of the invention shall be illustrated in an exemplary fashion by 
means of the following drawings. 

[0043] Figure 1 illustrates a schematic side view of an apparatus according to the 
invention for application and monitoring of an adhesive trail. 

[0044] Figure 2 illustrates a perspective view of an apparatus according to the 
invention of Figure 1 . 

[0045] Figure 3 illustrates a travel path of the apparatus according to the invention 
for application and monitoring of an adhesive trail. 

[0046] Figure 4 illustrates another travel path of the apparatus according to the 
invention with regard to switching of a relevant camera. 

[0047] Figure 5 is a view of a single image composed of three image strips from 
three cameras for online monitoring. 
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[0048] Figure 6 illustrates the principles of the design of the software. 

[0049] Figure 7 illustrates a schematic view of a calibrating device according to 
the invention for calibrating individual cameras of an apparatus according to the 
invention for recognizing a structure to be applied onto a substrate. 

[0050] Figure 8 illustrates a top view onto a substrate with an adhesive trail 
applied with regard to monitoring the application. 

[0051] Figure 9 illustrates a top view with regard to analysis of the profile. 

[0052] Reference number 10 illustrated in Figure 1 indicates schematically an 
apparatus for the application and recognition of an adhesive trail 20. In the center of the 
apparatus is arranged an application facility 1 1 by means of which an adhesive trail 20 is 
applied onto a substrate 30 or onto a sheet of metal 31 proceeding from right to left in 
Figure 1. Three cameras 12, 13, 14 are arranged at equal distances from each other in a 
circle around the application facility 11, wherein each camera is directed at the 
application facility 11. As is evident from Figure 1 , the axial longitudinal axes of the 
three cameras 12, 13, 14 intersect the axial longitudinal axis of the application facility 1 1 
below the substrate 30 such that the focus of the individual cameras 12, 13, and 14 are 
arranged around the area of the application facility 1 1 , in particular, on a circular line. 

[0053] In the inspection of the adhesive, either the application facility 1 1 with the 

cameras 12, 13, and 14 or the substrate 30 is moved. The adhesive trail 20 is 

simultaneously applied to the substrate 30 by means of the application facility 11, 

whereby the cameras 12, 13, 14 monitor the applied structure. Thus, either the 

application facility 1 1 with the cameras 12, 13 and 14 or the substrate 30 can be moved, 

in order to apply the adhesive trail 20 onto the substrate 30 such as to follow a desired 

progression. By this means, the moving cameras 12, 13, and 14 can monitor, independent 

of the path of travel, the adhesive trail 20 at the time the adhesive trail 20 is being 

applied. In Figure 2, the adhesive trail 20 progresses from left to right and is indicated by 

a continuous line. The intended progression of the adhesive trail 20 is indicated by a 
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dashed line to the right of the application facility 11. 

[0054] Figure 3 illustrates the progression of the adhesive trail 20 as indicated by 
arrows 21 and 22, whereby the direction or field of view of the three individual cameras 
12, 13 and 14 are shown in three sites 23, 24 and 25. The field of view 23, 24, and 25 of 
the three individual cameras 12, 13 and 14 is indicated by a rectangle drawn with a 
continuous line24, a rectangle drawn with widely dashed lines 23, and a rectangle drawn 
with narrow dashed lines 25. As shown in from Figure 3, the direction of the individual 
fields of view 23, 24, and 25 of the cameras 12, 13 and 14 remains constant at all times 
whereby only the whole apparatus is moved along the adhesive trail 20 as indicated by 
arrows 21 and 22. 

[0055] Figure 4 illustrates another progression of an adhesive trail 20, whereby a 
field of view is active (e.g., shown as "aktiv") 27. The camera 12, 13 and 14 having the 
corresponding active field of view is shown as an "aktiv" rectangle 27 while traveling 
along the adhesive trail 20. 

[0056] Figure 5 illustrates three image strips 32, 33, and 34 that represent a 
relevant section or strip of an image of the three individual cameras 12, 13 and 14 (shown 
in Figure 1). Each cameral2, 13 and 14 records a strip of the image in order to reduce 
the amount of data accordingly such that the recording rate can be increased. These 
individual image strips 32, 33, and 34 of the three cameras 12, 13 and 14 are then joined 
into an image. The image recording occurs at defined fixed time intervals and is 
independent of the robot control of the application facility 1 1 . For example, the cameras 
12, 13 and 14 only record a strip of the image having an image height of approximately 
100 pixels (e.g., 100 image lines is used) instead of an image height of 450 pixels. By 
means of this partial scanning technique (i.e., partial reading-out of the image recording 
chip) only small data streams are generated such that the image recording rate can be 
increased several-fold. Therefore, the data analysis simultaneously captures the three 
image strips that are arranged one below the other of the individual cameras 12, 13 and 
14 synchronously and in parallel and then joins them into a single image. As a result, the 
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three image strips, are correctly arranged and assigned with regard to a location and a 
time relative to each other and without delay can be processed accordingly. The specific 
image recording technique facilitates simultaneous and parallel recording of individual 
camera 12, 13 and 14 that are stored as a sequence of images. Thus, the structure can be 
scanned once during the teach-in of a reference application structure. 

[0057] Once the images of the three cameras 12, 13 and 14 are stored in a 
sequence of images, a parameterization of the reference track is carried out as the 
subsequent step of teaching-in the reference adhesive trail. A robot travel path, a robot 
travel time, a direction, a width, and a quality of the adhesive trail 20 are utilized to 
determine the parameterization. A type of vector chain for the adhesive trail 20 results, 
which allows a high image recording rate to be attained via comparably short partial 
sections (e.g., between 1 and 3 mm). Vectorization has another advantage, in that, the 
adhesive trail 20, being in the form of a vector chain, can be stored in a camera- 
transcending global coordinate system. 

[0058] As shown in the bottom strip 34 of Figure 5, a circular line is arranged 
around the center of the application facility 11, and two edge points 21 and 22 of the 
adhesive trail 20 are arranged on the circular line. The circular line is subdivided into a 
plurality of angles to provide overlapping areas of the individual cameras 12, 13 and 14 
to facilitate a gapless coverage around the application facility 1 1 . A range of angles from 
-10° to 130° is assigned to a first camera. A range of angles from 110° to 250° is 
assigned to a second camera, and a range of angles from 230° to -10° is assigned to a third 
camera. A global coordinant system results for the three image strips, which can be 
provided as either Cartesian coordinants or polar coordinants. 

[0059] If the adhesive trail 20 progresses out of the field of view 23, 24 and 25 of 
a camera, the adhesive trail 20 is transiently in the overlapping area of the ranges of 
angles of the adjacent two cameras. If the adhesive trail 20 then progresses from the 
segment of the circular line of the one camera via the overlapping area to the segment of 
the circular line of another camera, an automatic switch is made from the one camera to 

12 



Atty. Dkt No.: QU01H10/P-WO2/US (22589- 14US) 

PATENT 

Express Mail No.: EB 270750868 US 

SUBSTITUTE SPECIFICATION CLEAN VERSION 

the other camera. This is illustrated in Figure 4 by means of the active fields of view 27 
of the individual cameras 12, 13 and 14. 

[0060] The individual cameras 12, 13 and 14 form a circular caliper whose center 
is formed by the application facility 11, whereby the search for the edges 21, 22 of the 
adhesive trail 20 proceeds on a circular line directly around the application facility 1 1 . 
The individual cameras 12, 13 and 14 are essentially directed at the application facility 1 1 
such that the axial longitudinal axes of the individual cameras 12, 13 and 14 intersect the 
longitudinal axis of the application facility 1 1 . 

[0061] A teach-in run or a teach-in of a reference adhesive trail is described in the 
following paragraphs. 

[0062] The teach-in process of the reference adhesive trail can be started by a user 
by means of a mouse-click by selecting a track that indicates the position of the adhesive 
trail 20. This is sufficient for fully automatic recognition of position and direction of the 
adhesive trail 20 in the subsequent camera images because the image recording rate is 
sufficiently high and the individual images are recorded very shortly after one another 
(e.g., every 1 mm to 3 mm). From the starting point, the adhesive is scanned image by 
image, whereby the adhesive trail position and the adhesive trail angle detected in the 
current image are used as a priori knowledge for the upcoming image. The fact that the 
track radii usually exceed 20mm facilitates fully automatic capture of the adhesive trail 
20 without a human having to assess the image or the position of the adhesive trail 20. 
As a result, the search area can be limited which allows, by means of the high image 
recording rate, a determination of where the adhesive trail 20 will essentially progress to 
in the following image. 

[0063] Figure 6 illustrates the principles of the design of the software, whereby 
the teach-in run or teach-in run generates the image sequence. The image sequence, in 
turn, facilitates the automatic parameterization. This parameterization can be pre-set by 
the user, if applicable, and is used together with a progression file for inspection of an 
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applied adhesive trail 20 during the inspection run. 

[0064] The online monitoring of an applied adhesive trail 20 is described in the 
following paragraphs. The application facility 11 (shown in Figure 1) applies the 
adhesive trail 20 onto the metal sheet 3 1 , whereby the application facility 1 1 is moved 
together with cameras 12, 13 and 14 over the metal sheet 31. However, a kinematic 
inversion is also feasible, for instance the metal sheet 31 is moved and the application 
facility 11, including the cameras 12, 13 and 14, is arranged in a fixed position. The 
applied adhesive trail 20 is determined and analyzed synchronously and in parallel by the 
cameras 12, 13, 14. The cameras 12, 13, and 14 are on the circular line of the circular 
caliper (shown in Figure 5), whereby each camera 12, 13, and 14 records only a strip of 
the image. The strips of images are joined into a single image that form a sequence of 
images. Thus, the image recording rate is increased in accordance with the data reduction 
attained by each camera 12, 13 and 14 recording only a strip of the image. The 
individual image strips in the joint image facilitate the synchronous and parallel as well 
as simultaneous capture of the three camera images. The individual images of the three 
cameras 12, 13 and 14 can be assigned directly as a function of location. As a result, 
online monitoring of the adhesive trail 20 is feasible in real-time and achieves high 
accuracy at high travel speeds due to the high image recording rate both in teaching-in a 
reference adhesive trail and in the inspection of the applied adhesive trail 20. The 
information concerning the adhesive trail 20 in the adhesive search area, the angle 
assignment of the sensor, the adhesive assessment, the robot travel path, and the robot 
travel time are summarized in a progression list. 

[0065] According to an embodiment of the present invention, an assessment 
function (e.g., in particular a fuzzy assessment) can be used to find the edges of the 
adhesive trail 20. In order to determine and assess the adhesive trail 20, the following 
parameters described below are included in the calculation of a fuzzy assessment. 

[0066] A width of a pair of edges (e.g., an edge 1 : a left edge of the adhesive 
trail; an edge 2: a right edge of the adhesive trail), a mean gray scale value of the 
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projected gray scale value profile between the pair of edges, an edge contrast (e.g., a 
geometric mean of amplitudes of the two edges), and a progression of position (e.g., 
directed deviation of a center between the two adhesive edges from a center of the search 
area, in pixels). By means of this plurality of parameters and the use of the fuzzy 
assessment function, the adhesive trail 20 can be automatically recognized in a reliable 
fashion and described very accurately. 

[0067] The illumination module (not shown) for the apparatus according to the 
invention comprises light emitting diodes (LEDs), in particular infrared LEDs, ultra- 
violet LEDs (UV LEDs) or Red-Green-Blue LEDs (RGB LEDs). In order to attain a 
high contrast in an image recording, the LEDs can be flashed. For instance short, strong 
pulses of current on the order of 1.0 to 0.01 ms can be applied to the diodes. Thus, the 
light-emitting diodes are capable of emitting light of various colors. The advantage is 
that a sensor design can be switched to other types of adhesives or colors of adhesives 
without reconfiguration. 

[0068] Figure 7 illustrates a calibration facility 40 in the form of a circular 
calibrating disc. The calibration facility 40 assigns the individual cameras 12, 13 and 14 
a scaling factor, an angle assignment, and a center as well as a radius of a search circle. 
The calibrating disc consists of individual form elements 41 (e.g. shown as dots) that are 
arranged on a circular line at an angle distance of approximately 1 0° Moreover, marker 
sites 42 are arranged at an equal distance from each other in order to calibrate three 
cameras 12, 13 and 14. A compensation calculation is used to calculate from the 
coordinates of the centers of the individual dots, on the one hand, the scaling factors of 
the individual cameras 12, 13 and 14 and, on the other hand, the center as well as the 
radius of the search area. The marker sites at angles of 0°, 120°, and 240° in the global 
coordinate system allow the angle assignment and the corresponding fields of view of the 
individual cameras 12, 13 and 14 to be determined. The field of view of the individual 
cameras 12, 13 and 14 is indicated, in particular, by the three rectangles 43, 44, and 45 
(shown in Figure 7). The form elements 41 can correspond to the circular line of the 
circular caliper for detection of the adhesive trail 20. 

15 



Atty. Dkt. No.; QU01H10/P-WO2/US (22589-14US) 

PATENT 

Express Mail No.: EB 270750868 US 

SUBSTITUTE SPECIFICATION CLEAN VERSION 

[0069] Figure 8 illustrates the application facility 11, whereby the strips 31, 32, 
and 33 around the application facility 11 each are shown by dashed lines that represent 
the read-out area of the individual cameras 12, 13 and 14. Two active cameras monitor 
the adhesive trail 20, such that the adhesive trail 20 is monitored in the overlapping area 
of the strips 32 and 33. If the progression of the adhesive trail 20 relative to the 
application facility 11 changes, only one of the cameras 12, 13 and 14 becomes active, 
and an essentially circular caliper (not shown) that is arranged to be concentric around the 
application facility 1 1 is used. 

[0070] According to this embodiment, the circular caliper is formed by multiple 
cameras 12, 13 and 14 that are arranged around the application facility 11, but, in 
particular, can be attached at a different radius from the center of the application facility 
11. For an inspection of an application structure or adhesive trail 20, the cameras 12, 13 
and 14 are directed at a circle or circular line whose center coincides with the center of 
the application facility 1 1 . The optical detection of the adhesive trail 20 then proceeds on 
this circular line. 

[0071] Figure 9 illustrates a three-dimensional profile analysis by means of a 
projection to provide for a positional correction of the application facility 11. For reasons 
of clarity of presentation, Figure 9 illustrates two camera fields of view 51, 52 indicated 
by the dashed lines. In the overlapping area of the two camera fields of view 51, 52 are 
shown a plurality of laser lines 60 that are used for profile analysis with regard to the 
width and contour of structure lines. The laser lines are also used for generation of soft 
contours. The laser lines 60 are generated by a projection facility that can, for example, 
be arranged on the optical sensor with three cameras 12, 13 and 14. Moreover, the 
projection facility can also be arranged directly on the application facility 11. The sensor 
with the three cameras 12, 13 and 14 is shown schematically by the circle 70. The laser 
lines 60 or laser strips projected onto the component 30 or metal sheet 31 highlight 
contours on the component that cannot be used for three-dimensional analysis by 
conventional image processing. Artificial features are generated by means of the laser 
lines 60 on the component and can subsequently be analyzed by means of image 
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processing utilizing stereometry. Thus, Figure 9 illustrates the principle of three- 
dimensional positional recognition prior to the application of a sealing agent where no 
hard, analyzable features are present. 
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